1. Field of the Invention
The present invention relates to a face recognition apparatus and method and computer program for identifying a face included in a captured image and, in particular, to a face recognition apparatus and method and computer program for continuously performing face recognition in a moving image.
More particularly, the present invention relates to a face recognition apparatus and method and computer program for stably identifying the same person in the whole of a moving image and, in particular, to a face recognition apparatus and method and computer program for continuously identifying the same person under circumstances where the person's face moves up or down, or to the left or right, or the size of the face varies every frame.
2. Description of the Related Art
Face recognition technology is widely applicable to human-machine interfaces, e.g., an individual identification system and a sex determination system which impose no burden on users. Recent digital cameras have face recognition functions. Those cameras can identifies an object and perform various camera controls, such as automatic focusing, automatic exposure, and image quality adjustment, based on the result of identification.
Most related-art methods for face recognition perform face detection and identification using frames as still images. Typically, face recognition is performed using full-face images. Face identification is to determine whether a registered face image matches an input face image. The registered face image is preferably compared to the input face image showing a face with the same orientation and size as those in the registered face image. On the basis of the result of identification of an object, many captured images can be classified, organized, and associated with one another. One of possible applications of this technology is to assign levels of importance to respective persons in a captured image and control camerawork while locking focus and exposure on a main person assigned with the highest level of importance, or perform image processing, such as image quality adjustment, suitable for the main person.
Japanese Unexamined Patent Application Publication No. 2002-333652 discloses an imaging apparatus for previously recording information about features of an object, detecting a face in image data, detecting an object with features matching the predetermined features among the detected faces, and focusing on the detected object. This imaging apparatus records levels of importance in addition to information about features of objects. When a plurality of objects (persons) are present in captured image data, the imaging apparatus performs whether the features of an object match the predetermined features in order of decreasing importance.
Digital video cameras employing the above-described face recognition technology can similarly perform various camera controls, such as automatic focusing, automatic exposure, and image quality adjustment, on moving images and can also use information related to a specified person as information for video data management. Those digital video cameras can be applied to security systems for surveillance or reconnaissance on a person.
When face recognition is continuously performed in a moving image, the orientation of the face of an object or the size of the face may vary with the motion of the object. In other words, the object's face may move up or down, or to the left or right every frame or the size of the face may vary every frame. Accordingly, face identification every frame will probably lead to misidentification. For example, a person is erroneously identified as another person even though the same person is captured in a moving image. Alternatively, different persons are incorrectly identified as the same person. Furthermore, the effects of surrounding environment, including illumination and background, associated with the motion of a person produce a change in video image. The change causes noise in image recognition processing.
The results of face identification over the whole of a moving image include frames (or fields) in which a person can be correctly identified and frames (or fields) in which misidentification is performed. In many cases, the identity is not confirmed stably. When the result of face recognition differs from frame to frame in a moving image captured through, for example, a video camera, various camera controls, such as automatic focusing, automatic exposure, and image quality adjustment, are unstabilized.
Face recognition is broadly divided into two modes, i.e., a learning mode and a recognition mode. In the learning mode, the face of an object (person) is registered. In the recognition mode, a detected face is compared to the registered face to recognize the face, i.e., identify the person. However, if an old registered face is continuously used in a moving image stream, it becomes difficult to respond to a change in the orientation or size of a detected face caused by the motion of an object (person). Unfortunately, the probability of misidentification gradually increases.